{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "---\n",
    "sidebar_label: Google Cloud Vertex AI\n",
    "keywords: [gemini, vertex, ChatVertexAI, gemini-pro]\n",
    "---"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ChatVertexAI\n",
    "\n",
    "Note: This is separate from the Google PaLM integration. Google has chosen to offer an enterprise version of PaLM through GCP, and this supports the models made available through there. \n",
    "\n",
    "By default, Google Cloud [does not use](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance#foundation_model_development) customer data to train its foundation models as part of Google Cloud`s AI/ML Privacy Commitment. More details about how Google processes data can also be found in [Google's Customer Data Processing Addendum (CDPA)](https://cloud.google.com/terms/data-processing-addendum).\n",
    "\n",
    "To use `Google Cloud Vertex AI` PaLM you must have the `google-cloud-aiplatform` Python package installed and either:\n",
    "- Have credentials configured for your environment (gcloud, workload identity, etc...)\n",
    "- Store the path to a service account JSON file as the GOOGLE_APPLICATION_CREDENTIALS environment variable\n",
    "\n",
    "This codebase uses the `google.auth` library which first looks for the application credentials variable mentioned above, and then looks for system-level auth.\n",
    "\n",
    "For more information, see: \n",
    "- https://cloud.google.com/docs/authentication/application-default-credentials#GAC\n",
    "- https://googleapis.dev/python/google-auth/latest/reference/google.auth.html#module-google.auth\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!pip install -U google-cloud-aiplatform"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chat_models import ChatVertexAI\n",
    "from langchain.prompts import ChatPromptTemplate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content=\" J'aime la programmation.\")"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "system = \"You are a helpful assistant who translate English to French\"\n",
    "human = \"Translate this sentence from English to French. I love programming.\"\n",
    "prompt = ChatPromptTemplate.from_messages([(\"system\", system), (\"human\", human)])\n",
    "\n",
    "chat = ChatVertexAI()\n",
    "\n",
    "chain = prompt | chat\n",
    "chain.invoke({})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we want to construct a simple chain that takes user specified parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content=' プログラミングが大好きです')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "system = (\n",
    "    \"You are a helpful assistant that translates {input_language} to {output_language}.\"\n",
    ")\n",
    "human = \"{text}\"\n",
    "prompt = ChatPromptTemplate.from_messages([(\"system\", system), (\"human\", human)])\n",
    "\n",
    "chain = prompt | chat\n",
    "\n",
    "chain.invoke(\n",
    "    {\n",
    "        \"input_language\": \"English\",\n",
    "        \"output_language\": \"Japanese\",\n",
    "        \"text\": \"I love programming\",\n",
    "    }\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-06-17T21:09:25.423568Z",
     "iopub.status.busy": "2023-06-17T21:09:25.423213Z",
     "iopub.status.idle": "2023-06-17T21:09:25.429641Z",
     "shell.execute_reply": "2023-06-17T21:09:25.429060Z",
     "shell.execute_reply.started": "2023-06-17T21:09:25.423546Z"
    },
    "tags": []
   },
   "source": [
    "## Code generation chat models\n",
    "You can now leverage the Codey API for code chat within Vertex AI. The model name is:\n",
    "- codechat-bison: for code assistance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " ```python\n",
      "def is_prime(n):\n",
      "    if n <= 1:\n",
      "        return False\n",
      "    for i in range(2, n):\n",
      "        if n % i == 0:\n",
      "            return False\n",
      "    return True\n",
      "\n",
      "def find_prime_numbers(n):\n",
      "    prime_numbers = []\n",
      "    for i in range(2, n + 1):\n",
      "        if is_prime(i):\n",
      "            prime_numbers.append(i)\n",
      "    return prime_numbers\n",
      "\n",
      "print(find_prime_numbers(100))\n",
      "```\n",
      "\n",
      "Output:\n",
      "\n",
      "```\n",
      "[2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97]\n",
      "```\n"
     ]
    }
   ],
   "source": [
    "chat = ChatVertexAI(\n",
    "    model_name=\"codechat-bison\", max_output_tokens=1000, temperature=0.5\n",
    ")\n",
    "\n",
    "message = chat.invoke(\"Write a Python function to identify all prime numbers\")\n",
    "print(message.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Asynchronous calls\n",
    "\n",
    "We can make asynchronous calls via the Runnables [Async Interface](/docs/expression_language/interface)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# for running these examples in the notebook:\n",
    "import asyncio\n",
    "\n",
    "import nest_asyncio\n",
    "\n",
    "nest_asyncio.apply()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content=' Why do you love programming?')"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "system = (\n",
    "    \"You are a helpful assistant that translates {input_language} to {output_language}.\"\n",
    ")\n",
    "human = \"{text}\"\n",
    "prompt = ChatPromptTemplate.from_messages([(\"system\", system), (\"human\", human)])\n",
    "chain = prompt | chat\n",
    "\n",
    "asyncio.run(\n",
    "    chain.ainvoke(\n",
    "        {\n",
    "            \"input_language\": \"English\",\n",
    "            \"output_language\": \"Sanskrit\",\n",
    "            \"text\": \"I love programming\",\n",
    "        }\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Streaming calls\n",
    "\n",
    "We can also stream outputs via the `stream` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " The five most populous countries in the world are:\n",
      "1. China (1.4 billion)\n",
      "2. India (1.3 billion)\n",
      "3. United States (331 million)\n",
      "4. Indonesia (273 million)\n",
      "5. Pakistan (220 million)"
     ]
    }
   ],
   "source": [
    "import sys\n",
    "\n",
    "prompt = ChatPromptTemplate.from_messages(\n",
    "    [(\"human\", \"List out the 5 most populous countries in the world\")]\n",
    ")\n",
    "\n",
    "chat = ChatVertexAI()\n",
    "\n",
    "chain = prompt | chat\n",
    "\n",
    "for chunk in chain.stream({}):\n",
    "    sys.stdout.write(chunk.content)\n",
    "    sys.stdout.flush()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  },
  "vscode": {
   "interpreter": {
    "hash": "cc99336516f23363341912c6723b01ace86f02e26b4290be1efc0677e2e2ec24"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
